Signal processing method for producing interpolated signal values in an image signal

ABSTRACT

The signal processing method of the present invention produces interpolated signal values in an image signal by an inverse discrete cosine transformation with a set of frequency coefficients by decreasing an interval for reproduction of pixel signal values, wherein a set of frequency coefficients is provided by a discrete cosine transformation and is compensated for the frequency response caused by dividing pixels. According to the invention, an image signal similar to that obtained with a solid-state imaging device constructed by divided pixels is provided. Hence, even when the number of horizontal pixels and/or the number of vertical pixels of the solid-state imaging device are a half of that of a display apparatus used, an image signal suitable to the display apparatus can be produced.

CROSS-REFERENCE TO RELATED APPLICATION

Not Applicable

BACKGROUND OF THE INVENTION

1. Field of the Invention

This invention relates to an image signal processing method, and moreparticularly, to such image signal processing method for producinginterpolated signal values in an image signal.

2. Description of the Prior Art

The most high-resolution image signal that can be got from broadcastservices in Japan is called “full-high-vision” which is constructed byhorizontal 1920 pixels and vertical 1080 pixels. On the other hand, anadvanced image signal system called “4K” which is constructed by doublenumber of horizontal pixels and double number of vertical pixelscompared with “full-high-vision” has been developed recently. Inaddition, display apparatuses for image signals of “4K” are alreadyavailable. Thereby, some methods that convert the image signal of“full-high-vision” to the image signal of “4K” by interpolating pixelsare proposed.

Conventional methods for providing interpolated pixel signal values inan image signal are well known as the bi-linear interpolating methods orthe bi-cubic interpolating methods. Furthermore, methods that produceinterpolated pixel signal values in an image signal by a discrete cosinetransformation and an inverse discrete cosine transformation aredisclosed in U.S. Pat. No. 5,168,375 issued Dec. 1, 1992 to M. Reisch etal. and U.S. Pat. No. 7,139,443 issued Nov. 22, 2006 to N. Ozawa.

The methods disclosed in those patents are accomplished by extracting ablock of pixel signal values from an image signal, providing a set offrequency coefficients by a discrete cosine transformation with theblock of pixel signal values, and providing interpolated pixel signalvalues by an inverse discrete cosine transformation, decreasing thesampling interval from that of original sampling locations, with the setof frequency coefficients. Since the set of frequency coefficientsgenerated with the original pixel signal values is used for the inversediscrete cosine transformation, the frequency content of the imagesignal is not affected by the interpolated pixel signal values.

In addition, it is disclosed in U.S. Pat. No. 5,168,375 that thefiltering function for an image signal produced by the inverse discretecosine transformation is accomplished by modifying the set of frequencycoefficients. Thus, the filtering function is accomplished by asimplified process when interpolated pixel signal values are produced bythe discrete cosine transformation and the inverse discrete cosinetransformation. On the other hand, since the filtering function in thespatial domain is accomplished by convolving a block of pixel signalvalues with a filter kernel, the procedure of the filtering function inthe spatial domain is complicated.

Moreover, it is disclosed in U.S. Pat. No. 7,139,443 that aninterpolated pixel signal value produced by the discrete cosinetransformation and the inverse discrete cosine transformation isprovided by a linear combination of each pixel signal value in a blockof pixel signal values. By applying these methods, a procedure producinginterpolated signal values becomes more simplified.

Meanwhile, most of image capturing apparatuses, not only video camerasand digital still cameras for consumer but also television cameras forbroadcast, are fabricated with solid-state imaging devices recently. Inan image capturing apparatus employing a solid-state imaging device, animage signal captured by pixels arranged on the imaging area of thesolid-state imaging device is converted in an image signal formatcorresponding to a display apparatus used. Hence, when a televisioncamera for the image signal of “full-high-vision” is fabricated by asolid-state imaging device comprising horizontal 1920 pixels andvertical 1080 pixels, each pixel of the solid-state imaging devicecorresponds to each pixel of the display apparatus for the image signalof “full-high-vision”.

Therefore, the conversion of the image signal of “full-high vision” tothe image signal of “4K” corresponds to dividing each pixel of thesolid-state imaging device in a half of the original pixels in thehorizontal direction and the vertical direction. Then, when the imagesignal of “4K” is produced by interpolating the image signal of“full-high vision” provided by a solid-state imaging device, the imagesignal of “4K” should be compensated by the frequency response caused bydividing pixels of the solid-state imaging device. However, a filteringcharacteristic suitable to compensate the frequency response caused bydividing pixels of the solid-state imaging device is not disclosed inthe prior art.

BRIEF SUMMARY OF THE INVENTION

The object of the present invention is to provide a signal processingmethod for producing interpolated signal values in an image signal bythe discrete cosine transformation and the inverse discrete cosinetransformation, wherein the signal processing method comprisescompensating for the frequency response caused by dividing pixels of thesolid-state imaging device.

The object of the invention is achieved by extracting a block of pixelsignal values from the pixel signal values of an image signal, producinga set of frequency coefficients by an orthogonal transformation with theblock of pixel signal values, providing a set of compensatingcoefficients, converting the set of frequency coefficients to a set ofcompensated frequency coefficients by multiplying each compensatingcoefficient of the set of compensating coefficients to a correspondingfrequency coefficient of the set of frequency coefficients, producing atleast one represented pixel signal value that corresponds to anarbitrary location between adjacent pixel signal values of the block ofpixel signal values by an inverse transformation of the orthogonaltransformation with the set of compensated frequency coefficients bydecreasing an interval for reproduction of pixel signal values, andapplying the represented pixel signal value to a converted image signalas a pixel signal value corresponding to the arbitrary location of theconverted image signal. Here, each compensating coefficient of the setof compensating coefficients corresponds to a ratio of a frequencyresponse caused by sampling by the pixels of the converted image signalto a frequency response caused by sampling by the pixels of the imagesignal at corresponding frequency.

According to this method, an image signal similar to that obtained witha solid-state imaging device constructed by divided pixels is provided.Consequently, even when the number of horizontal pixels and/or thenumber of vertical pixels of the solid-state imaging device are a halfof that of a display apparatus used, an image signal suitable to thedisplay apparatus can be produced.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention is described with reference to the drawings, wherein:

FIG. 1 is a block diagram showing the structure of an imaging apparatusaccording to a preferred embodiment of the present invention;

FIG. 2 is a block diagram showing the structure of a block-extractingprocessor usable in the imaging apparatus shown in FIG. 1;

FIG. 3 is an illustration of the block of pixel signal values;

FIG. 4A is an illustration of adjacent four pixels not divided;

FIG. 4B is an illustration of divided pixels corresponding to the pixelsshown in FIG. 4A;

FIG. 5 is a magnitude spectrum representing the ratio of the frequencyresponse caused by sampling by the original pixels to a frequencyresponse caused by sampling by the divided pixels;

FIG. 6 is an example of the compensating coefficients usable in thefrequency response compensator;

FIG. 7 is a block diagram showing the structure of an imaging apparatusaccording to another preferred embodiment of the present invention;

FIG. 8A is an illustration of adjacent four pixels not divided, whereinthe aperture ratio of pixel is 81%;

FIG. 8B is an illustration of divided pixels corresponding to the pixelsshown in FIG. 8A;

FIG. 9 shows formula 1 and formula 7.

DETAILED DESCRIPTION OF THE INVENTION

Referring now to the drawings, exemplary embodiments of the inventionwill be described. An example of the video apparatus in which theinvention is applied is shown in FIG. 1.

In the video apparatus shown in FIG. 1, reference numeral 1 represents aimaging apparatus. In the imaging apparatus 1, a solid-state imagingdevice 102 converts an image captured by an optical unit 101 including alens to an image signal. Here, the image signal is converted to a redsignal, a green signal, and a blue signal by a color signal processor103. There, each signal of the red signal, the green signal, and theblue signal is assumed to comprise pixel signal values corresponding toall pixels of the solid-state imaging device 102.

Next, reference number 2 represents an interpolating processor inFIG. 1. In the interpolating processor 2, each of the red signal, thegreen signal, and the blue signal is applied to a block-extractingprocessor 201 respectively. The block-extracting processor 201 extractsa block of pixel signal values corresponding to the block of 64 pixelsfabricated by 8 adjoined pixels in a horizontal direction and 8 adjoinedpixels in a vertical direction, and outputs pixel signal valuescorresponding to all pixels in the block of pixels simultaneously. Theblock of pixel signal values from the block-extracting processor 201 areapplied to a discrete cosine transformer 202, and are converted to a setof frequency coefficients, being composed by 8 horizontal frequencycoefficients and 8 vertical frequency coefficients, by a discrete cosinetransformation.

The set of frequency coefficients outputted from the discrete cosinetransformer 202 is applied to a frequency response compensator 203, andis compensated by multiplying each frequency coefficient with thecompensating coefficient that corresponds to the frequency responsecaused by dividing pixels of the solid-state imaging device. The set ofcompensated frequency coefficients outputted from the frequency responsecompensator 203 is applied to an inverse discrete cosine transformer204. In the inverse discrete cosine transformer 204, interpolated pixelsignal values are generated by an inverse discrete cosine transformationwith the set of compensated frequency coefficients, by setting thesampling interval to a half of that of original pixels. Thus, theinterpolated pixel signal values corresponding to locations between theadjacent original pixels are produced. As the result, number of thepixels is doubled compared with the original pixels in the horizontaldirection and the vertical direction.

Next, the pixel signals generated by the inverse discrete cosinetransformer 204 are applied to an arranging processor 205, and arearranged to an image signal corresponding to a display apparatus thathas double pixels in the horizontal direction and the vertical directioncompared with the solid-state imaging device 102. The image signaloutputted from the arranging processor 205 in the interpolatingprocessor 2 is applied to following display apparatus or transmissionapparatus that is not shown in FIG. 1.

Referring now to the drawings, the procedure of the interpolatingprocessor 2 will be described in detail, especially being focused to thecompensating coefficients employed in the frequency response compensator203. In the following descriptions, assuming that the green signal, thered signal, and blue signal are processed by same procedure in theinterpolating processor 2, only the procedure for the green signal isexplained.

In the interpolating processor 2, the green signal generated by thecolor signal processor 103 in the imaging apparatus 1 is applied to theblock-extracting processor 201. By the block-extracting processor 201, ablock of pixel signal values corresponding to 64 pixels constructed by 8adjoined pixels in the horizontal direction and 8 adjoined pixels in thevertical direction is extracted from the input signal so that all pixelsignal values in the block of pixel signal values are outputtedsimultaneously. An exemplary embodiment of the block-extractingprocessor 201 is shown in FIG. 2. As shown in FIG. 2, an input signal isdelayed by seven 1H delayers 301 chained in serially, the 1H delayerdelaying the input signal for one horizontal scan period. Thus, 8 pixelsignal values corresponding to adjoined 8 pixels in the verticaldirection are outputted simultaneously. Furthermore, the input signal ofthe block-extracting processor 201 and output signals of seven 1Hdelayers 301 are delayed respectively by seven 1-pixel delayers 302chained in serially, the 1-pixel delayer delaying the input signal forone pixel sampling period. Thus, 8 pixel signal values corresponding toadjoined 8 pixels on a horizontal pixel line are outputtedsimultaneously. As the result, a block of pixel signal valuescorresponding to 64 pixels constructed by 8 adjoined pixels in thehorizontal direction and 8 adjoined pixels in the vertical direction areoutputted from the block-extracting processor 201 simultaneously.

An exemplified block of pixel signal values extracted by theblock-extracting processor 201 is shown in FIG. 3. In FIG. 3, “i”represents vertical location of the pixel, and “j” represents thehorizontal location of the pixel. Here, the pixel at the top left-handcorner corresponding to i=0 and j=0 is defined as “X(0,0)”. Further,“x(0,0)”, “x(0,1)”, etc. written in the grid mean pixel signal valuesgenerated by corresponding pixels.

The block of pixel signal values provided by the block-extractingprocessor 201 is applied to a discrete cosine transformer 202, and isconverted to a set of frequency coefficients by a discrete cosinetransformation defined by formula 1 shown in FIG. 9. Assuming N=8 informula 1, the set of frequency coefficients comprises 64 frequencycoefficients, corresponding to horizontal 8 frequencies represented withv=0 to 7 and vertical 8 frequencies represented with u=0 to 7. Here,“F(u,v)” is defined as the frequency coefficient corresponding tohorizontal frequency v and vertical frequency u.

The set of frequency coefficients is applied to the frequency responsecompensator 203, and is compensated with the frequency response causedby dividing pixels of the solid-state imaging device. Next, thefrequency response caused by dividing pixels is explained.

An example of four pixels adjacent in the horizontal direction and thevertical direction is represented in FIG. 4A. Here, the aperture ratioof the pixel is assumed to be approximately 100%. The aperture ratiobeing 100% means a situation in which all the light injected to a pixelis employed to generate the pixel signal, and can be realized by aback-illuminated imaging device or by an imaging device which combined amicro lens on each pixel.

In FIG. 4A, pixel signal values generated by X(n,m), X(n,m+1), X(n+1,m),and X(n+1,m+1) are outputted as x(n,m), x(n,m+1), x(n+1,m), andx(n+1,m+1) which are detected at A, B, C, and D respectively. Here, thepixels shown in FIG. 4A are divided so as to double the number of pixelsin the horizontal direction and the vertical direction, wherein thedivided pixels comprise the pixels that generate pixel signal valuescorresponding to A, B, C, and D. Namely, the centers of the pixelsprovided by dividing are allocated at A, B, C, D, and their middlepoints as shown in FIG. 4B.

In FIG. 4B, the pixels corresponding to A, B, C, and D are defined as X2(2n,2m), X2(2n,2(m+1)), X2(2(n+1),2m), and X2(2(n+1),2(m+1))respectively. Likewise, the pixel corresponding to the middle point of Aand B is defined as X2(2n,2m+1), and the pixel corresponding to themiddle point of A and C is defined as X2(2n+1,2m). Similarly, theupper-left pixel of the pixel X2(2n,2m) is defined as X2(2n−1,2m−1), theupper pixel of the pixel X2(2n,2m) is defined as X2(2n−1,2m), theupper-right pixel of the pixel X2(2n,2m) is defined as X2(2n−1,2m+1),the left pixel of the pixel X2 (2n,2m) is defined as X2(2n,2m−1), theright pixel of the pixel X2(2n,2m) is defined as X2(2n,2m+1), thelower-left pixel of the pixel X2(2n,2m) is defined as X2(2n+1,2m−1), thelower pixel of the pixel X2(2n,2m) is defined as X2(2n+1,2m), and thelower-left pixel of the pixel X2(2n,2m) is defined as X2(2n+1,2m+1)respectively as shown in FIG. 4B.

Comparing FIG. 4B with FIG. 4A, it is known that the pixel X(n,m)includes all of the pixel X2(2n,2m) and a half of the adjacent pixels,X2(2n−1,2m), X2(2n,2m−1), X2(2n,2m+1), and X2 (2n+1,2m). Further, it isknown that the pixel X(n, m) also includes a quarter of the pixels,X2(2n−1,2m−1), X2(2n−1,2m+1), X2(2n+1,2m−1), and X2(2n+1,2m+1), whichexist in the slanted direction. Thus, when the pixel signal valuegenerated by X2 (2n, 2m) is defined as x2 (2n, 2m), the pixel signalvalue x(n,m) generated by X(n,m) are represented by formula 2,

$\begin{matrix}{{x\left( {n,m} \right)} = {{x\; 2\left( {{2n},{2m}} \right)} + {\left\{ {{x\; 2\left( {{{2n} - 1},{2m}} \right)} + {x\; 2\left( {{2n},{{2m} - 1}} \right)} + {x\; 2\left( {{2n},{{2m} + 1}} \right)} + {x\; 2\left( {{{2n} + 1},{2m}} \right)}} \right\}*{1/2}} + {\left\{ {{x\; 2\left( {{{2n} - 1},{{2m} - 1}} \right)} + {x\; 2\left( {{{2n} - 1},{{2m} + 1}} \right)} + {x\; 2\left( {{{2n} + 1},{{2m} - 1}} \right)} + {x\; 2\left( {{{2n} + 1},{{2m} + 1}} \right)}} \right\}*{1/4.}}}} & (2)\end{matrix}$

For simplification, a pixel signal is assumed to be a one-dimensionalsignal here. Namely, the pixel signal value x2(2n,2m) is defined asformula 3,x2(2n,2m)=exp(j2*PHI*f*2m*dx),  (3)which represents that the pixel signals vary into a sinusoidal wave onlyhorizontally. Here, “PHI” means the constant pi, “f” means arbitraryfrequency, and “dx” is a size of the pixels divided. Consequently,formula 2 is rewritten to formula 4,

$\begin{matrix}{{x\left( {n,m} \right)} = {{{\exp\left( {j\; 2*{PHI}*f*2m*{dx}} \right)} + {\left\{ {{\exp\left( {j\; 2*{PHI}*f*2m*{dx}} \right)} + {\exp\left( {j\; 2*{PHI}*f*\left( {{2m} - 1} \right)*{dx}} \right)} + {\exp\left( {j\; 2*{PHI}*f*\left( {{2m} + 1} \right)*{dx}} \right)} + {\exp\left( {j\; 2*{PHI}*f*2m*{dx}} \right)}} \right\}{1/2}} + {\left\{ {{\exp\left( {j\; 2*{PHI}*f*\left( {{2m} - 1} \right)*{dx}} \right)} + {\exp\left( {j\; 2*{PHI}*f*\left( {{2m} + 1} \right)*{dx}} \right)} + {\exp\left( {j\; 2*{PHI}*f*\left( {{2m} - 1} \right)*{dx}} \right)} + {\exp\left( {j\; 2*{PHI}*f*\left( {{2m} + 1} \right)*{dx}} \right)}} \right\}*{1/4}}} = {2*{\exp\left( {{j2}*{PHI}*f*2m*{dx}} \right)}*{\left\{ {1 + {{\exp\left( {{- j}\; 2*{PHI}*f*{dx}} \right)}/2} + {{\exp\left( {j\; 2*{PHI}*f*{dx}} \right)}/2}} \right\}.}}}} & (4)\end{matrix}$Furthermore, by applyingexp(jx)=cos(x)+j sin(x),formula 4 is rewritten to formula 5,x(n,m)=4*exp(j2*PHI*f*2m*dx)*½*{1+cos(2*PHI*f*dx)}.  (5)

A forward part of right side of formula 5, exp(j2*PHI*f*2m*dx),represents the original signal that has constant amplitude and variesinto a sinusoidal wave. On the other hand, a latter part of right sideof formula 5, ½*{1+cos(2*PHI*f*dx)}, represents the frequency responsecorresponding to the ratio of the image signal generated by sampling bythe original pixels to the image signal generated by sampling by thedivided pixels. In other words, the reciprocal of ½*{1+cos(2*PHI*f*dx)},2/{1+cos(2*PHI*f*dx)}, represents the frequency compensating coefficientcorresponding to the ratio of the image signal generated by sampling bythe divided pixels to the image signal generated by sampling by theoriginal pixels.

The frequency response represented by ½*{1+cos(2*PHI*f*dx)} is shown inFIG. 5. In FIG. 5, the horizontal axis represents the frequency of theimage signal, and the longitudinal axis represents the magnitude ofresponse. As shown in FIG. 5, the response at the sampling frequency ofthe original pixels which is represented by ½dx is zero, and theresponse at the nyquist frequency of the original pixels which isrepresented by ¼dx is ½. Furthermore, the nyquist frequency ¼dxcorresponds to v=8 in formula 1. Thus, the frequency response R(v)represented by v becomes as formula 6,

$\begin{matrix}\begin{matrix}{{R(v)} = {{1/2}*\left\{ {1 + {\cos\left( {2*{PHI}*\left( {{v/32}{dx}} \right)*{dx}} \right)}} \right\}}} \\{= {{1/2}*{\left\{ {1 + {\cos\left( {{PHI}*{v/16}} \right)}} \right\}.}}}\end{matrix} & (6)\end{matrix}$

Hence, in the frequency response compensator 203, the reciprocal of thefrequency response R(v) is multiplied to the frequency coefficientcorresponding to v in the set of the frequency coefficients provided bythe discrete cosine transformer 202. Consequently, the set of frequencycoefficients provided by a discrete cosine transformation with theoriginal pixel signal values is converted to the compensated frequencycoefficients that should be provided by a discrete cosine transformationwith the divided pixel signal values.

Though a processing in the horizontal direction is explained in theabove descriptions, a processing in the vertical direction is same asthat in the horizontal direction. The coefficients for the frequencycompensation calculated as the reciprocal of R(u,v) which is generatedby multiplication of R(u) and R(V) are shown in FIG. 6. In FIG. 6, thehorizontal grid location corresponds to horizontal frequency v and thevertical grid location corresponds to vertical frequency u. Thus, thevalue presented in the grid corresponding to horizontal frequency v andvertical frequency u means the compensating coefficient that is thereciprocal of R(u,v) and should be multiplied to the frequencycoefficient corresponding to horizontal frequency v and verticalfrequency u. Consequently, each compensating coefficient shown in FIG. 6is multiplied to corresponding frequency coefficient of the set offrequency coefficients provided by the discrete cosine transformer 202in the frequency response compensator 203.

Thereafter, the compensated frequency coefficients, F2(u,v), provided bythe frequency response compensator 203 is applied to the inversediscrete cosine transformer 204. In the inverse discrete cosinetransformer 204, pixel signals whose interval is a half of that of theoriginal pixels in the horizontal direction and the vertical directionare generated by an inverse discrete cosine transformation.Consequently, when x2(i,j) defined by formula 7 shown in FIG. 9 iscalculated by applying from 0 up to 2N−1 for i and j, pixel signalvalues that should be outputted by a solid-state imaging devicecomprising pixels divided in a half horizontally and vertically comparedwith original pixels are generated.

Thereafter, the interpolated pixel signal values generated by theinverse discrete cosine transformer 204 are arranged into an imagesignal that comprises horizontal scan lines doubled and horizontalpixels doubled compared with the original pixels by the arrangingprocessor 205. Further, the image signal arranged by the arrangingprocessor 205 is outputted as the output image signal of theinterpolating processor 2.

Another example of the structure of a video apparatus according to anembodiment of the present invention is shown in FIG. 7. In FIG. 7, theimaging apparatus 1 is assumed to be same as that shown in FIG. 1.

The green signal, the red signal, and the blue signal, being provided bythe color signal processor 103 in the imaging apparatus 1, are appliedto an interpolating processor 4. In the interpolating processor 4, eachof the green signal, the red signal, and the blue signal is applied tothe block-extracting processor 201 respectively. Here, the function ofthe block-extracting processor 201 is same as that in the interpolatingprocessor 2 shown in FIG. 1.

On the other hand, the block of pixel signal values extracted by theblock-extracting processor 201 is applied to a filtering processor 206in the interpolating processor 4.

As disclosed in U.S. Pat. No. 7,139,443, when an interpolated pixelsignal value is generated by a discrete cosine transformation and aninverse discrete cosine transformation, the interpolated pixel signalvalue is composed by a linear combination of the pixel signal values inthe block of pixel signal values. Thus, the interpolated pixel signalvalue can be obtained by a two-dimensional filter processing to theblock of pixel signal values applied to the discrete cosine transformer202. Furthermore as mentioned above, the processing of the frequencyresponse compensator 203 in the interpolating processor 2 shown in FIG.1 is accomplished by multiplying each compensating coefficient tocorresponding frequency coefficient in the set of frequencycoefficients. Thus, the interpolated pixel signal value compensated bythe frequency response compensator 203 can be obtained by a linearcombination of the pixel signal values in the block of pixel signalvalues, and can be obtained by a two-dimensional filter processing tothe block of pixel signal values.

Hence, when the filtering processor 206 is assumed to comprise sets offiltering coefficients, wherein each set of filtering coefficientscorresponds to processing for generation of each of interpolated pixelsignal values that are generated by the discrete cosine transformer 202,the frequency response compensator 203, and the inverse discrete cosinetransformer 204, the output signal values generated by the filteringprocessor 206 become same as that generated by the inverse discretecosine transformer 204 in FIG. 1.

Here, the set of filtering coefficients for the filtering processor 206can be obtained by the procedure disclosed in U.S. Pat. No. 7,139,443.That is, only one pixel signal value selected from the block of pixelsignal values arbitrarily is set to “1”, and other pixel signal valuesare set to “0”. Then, an interpolated pixel signal value correspondingto a predetermined location is generated by processing of the discretecosine transformer, the frequency response compensator, and the inversediscrete cosine transformer. Here, the interpolated pixel signal valueis defined as the filtering coefficient of the set of filteringcoefficients for the predetermined location, wherein the location of thefiltering coefficient in the set of filtering coefficients correspondsto the pixel which is set to “1” in the block of pixel signal values.

The output signal values generated by the filtering processor 206 areapplied to the arranging processor 205 like the output signal valuesgenerated by the inverse discrete cosine transformer 204 in theinterpolating processor 2 shown in FIG. 1 are, and are converted into animage signal that comprises horizontal scan lines doubled and horizontalpixels doubled compared with the original pixels. Further, the imagesignal generated by the arranging processor 205 in the interpolatingprocessor 4 is applied to following display apparatus or transmissionapparatus which is not shown in FIG. 7.

In the above descriptions, the procedure for the red signal or the bluesignal is assumed same as that of the green signal. However, when thecolor signal processor outputs a luminance signal and two chrominancesignals, the procedure employing the present invention may be employedonly for the luminance signal. Here, the chrominance signals may beinterpolated by a conventional method.

Further, though the imaging apparatus is followed by the interpolatingprocessor directly in the above descriptions, it is apparent that thetransmission apparatus could be inserted between the imaging apparatusand the interpolating processor.

Additionally, though the discrete cosine transformation is adopted as anorthogonal transformation in the above descriptions, it is apparent thatsame result is expected by employing the discrete sine transformation orthe hadamard transformation.

Furthermore, though the example in which the pixels are divided in thehorizontal direction and the vertical direction is explained in theabove descriptions, the inverse discrete cosine transformer may generatepixel signal values divided only in the horizontal direction or only inthe vertical direction corresponding to the display apparatus.

Further, the aperture ratio of the pixels of the solid-state imagingdevice is assumed to be approximately 100% in the above descriptions.However, even when the aperture ratio of the pixels is lower than 100%,the compensating coefficient of the frequency response compensator 203can be obtained as the reciprocal of the frequency response provided byformula 4, wherein the coefficient for adjacent pixels is amended tolower than ½ and the coefficient for pixels which exist in the slanteddirections is amended to lower than ¼.

For example, the aperture ratio is assumed to be 81% in which the lightinjected to a pixel area of 90% in the horizontal direction and in thevertical direction is used to generate the pixel signal. Additionally,it is assumed that the pixel size is 2dx in the horizontal direction andthe vertical direction as shown in FIG. 8A, and it is assumed that thelight injected to the surrounding portion of the pixel up to dx/10 fromthe circumference does not generate the pixel signal. Here, when thepixels are divided in same manner as shown in FIG. 4B, the dividedpixels are represented in FIG. 8B.

As shown in FIG. 8B, though the aperture ratio of the pixel X4(2n,2m) isstill 100%, the aperture ratio of the pixel X4(2n+1,2m) is 80% becausethe light injected to a pixel area of 20% which corresponds to dx/5 ofthe pixel size dx in the vertical direction is not used to generate thepixel signal. Also, the aperture ratio of the pixel X4(2n+1,2m+1) is 64%because the light injected to a pixel area of 20% in the horizontaldirection and the vertical direction is not used to generate the pixelsignal.

Here, formula 8 which represents the pixel signal x3(n,m) generated byX3(n,m) shown in FIG. 8A is defined instead of formula 2,

$\begin{matrix}{{{x\; 3\left( {n,m} \right)} = {{x\; 4\left( {{2n},{2m}} \right)} + {\left\{ {{x\; 4\left( {{{2n} - 1},{2m}} \right)} + {x\; 4\left( {{2n},{{2m} - 1}} \right)} + {x\; 4\left( {{2n},{{2m} + 1}} \right)} + {x\; 4\left( {{{2n} + 1},{2m}} \right)}} \right\}*{2/5}} + {\left\{ {{x\; 4\left( {{{2n} - 1},{{2m} - 1}} \right)} + {x\; 4\left( {{{2n} - 1},{{2m} + 1}} \right)} + {x\; 4\left( {{{2n} + 1},{{2m} - 1}} \right)} + {x\; 4\left( {{{2n} + 1},{{2m} + 1}} \right)}} \right\}*{4/25}}}},} & (8)\end{matrix}$and formula 9 which represents the frequency response R2(v) is definedinstead of formula 6,

$\begin{matrix}\begin{matrix}{{R\; 2(v)} = {{5/9}*\left\{ {1 + {{4/5}*{\cos\left( {2*{PHI}*{v/32}{dx}*{dx}} \right)}}} \right\}}} \\{= {{5/9}*{\left\{ {1 + {{4/5} \cdot {\cos\left( {{PHI}*{v/16}} \right)}}} \right\}.}}}\end{matrix} & (9)\end{matrix}$

When the aperture ratio in the horizontal direction and in the verticaldirection is defined as “a”, the frequency response R3(v) is representedby formula 10, wherein “a” is assumed to be from ½ up to 1,

$\begin{matrix}\begin{matrix}{{R\; 3(v)} = {{1/2}a*\left\{ {1 + {\left( {{2a} - 1} \right)*{\cos\left( {2*{PHI}*{v/32}{dx}*{dx}} \right)}}} \right\}}} \\{= {{1/2}a*{\left\{ {1 + {\left( {{2a} - 1} \right)*{\cos\left( {{PHI}*{v/16}} \right)}}} \right\}.}}}\end{matrix} & (10)\end{matrix}$

Thus, the compensating coefficient of the frequency response compensator203 is obtained as the reciprocal of R3(v) calculated by formula 10.

Furthermore, although the invention has been described as being realizedby hardware processing, it is apparent to one skilled in the art thatthe invention can be realized by software processing.

I claim:
 1. A method for processing an image signal wherein said imagesignal is consisted of a plurality of pixel signal values arrangedtwo-dimensionally, comprising: (a) extracting a block of pixel signalvalues from said pixel signal values; (b) producing a set of frequencycoefficients by an orthogonal transformation with said block of pixelsignal values; (c) providing a set of compensating coefficients; (d)converting said set of frequency coefficients to a set of compensatedfrequency coefficients by multiplying each compensating coefficient ofsaid set of compensating coefficients to a corresponding frequencycoefficient of said set of frequency coefficients; (e) producing atleast one represented pixel signal value that corresponds to a arbitrarylocation between adjacent pixel signal values of said block of pixelsignal values by an inverse transformation of said orthogonaltransformation with said set of compensated frequency coefficients bydecreasing an interval for reproduction of pixel signal values; and (f)applying said represented pixel signal value to a converted image signalas a pixel signal value corresponding to said arbitrary location of saidconverted image signal, wherein each compensating coefficient of saidset of compensating coefficients corresponds to a ratio of a frequencyresponse caused by sampling by the pixels of said converted image signalto a frequency response caused by sampling by the pixels of said imagesignal at corresponding frequency.
 2. A method according to claim 1wherein said orthogonal transformation is a discrete cosinetransformation.
 3. A method according to claim 1 wherein said ratio of afrequency response caused by sampling by the pixels of said convertedimage signal to a frequency response caused by sampling by the pixels ofsaid image signal at corresponding frequency is defined as2a/{1+(2a−1)*cos(2*PHI*f*dx)}, where “a” means the aperture ratio ofsaid pixel of said image signal of related direction and is defined tobe from ½ up to 1, “dx” means the size of said pixel of said convertedimage signal of related direction, “f” means said correspondingfrequency and is defined to be less than 1/(4dx), “PHI” means pi, and“*” means multiplication.
 4. A method according to claim 2 wherein saidratio of a frequency response caused by sampling by the pixels of saidconverted image signal to a frequency response caused by sampling by thepixels of said image signal at corresponding frequency is defined as2a/{1+(2a−1)*cos(2*PHI*f*dx)}, where “a” means the aperture ratio ofsaid pixel of said image signal of related direction and is defined tobe from ½ up to 1, “dx” means the size of said pixel of said convertedimage signal of related direction, “f” means said correspondingfrequency and is defined to be less than 1/(4dx), “PHI” means pi, and“*” means multiplication.
 5. A method for processing an image signalwherein said image signal is consisted of a plurality of pixel signalvalues arranged two-dimensionally, comprising: (a) extracting a block ofpixel signal values from said pixel signal values; (b) providing atleast one set of coefficients corresponding to a arbitrary locationbetween adjacent pixel signal values of said block of pixel signalvalues; (c) producing a represented pixel signal value by integratingall of pixel signal values in said block of pixel signal values, eachpixel signal value in said block of pixel signal values being multipliedby corresponding coefficient of said set of coefficients; and (d)applying said represented pixel signal value to a converted image signalas a pixel signal value corresponding to said arbitrary location of saidconverted image signal, wherein said providing at least one set ofcoefficients corresponding to said arbitrary location includes: defininga block of impulse signal values, said block of impulse signal valuescomprising a signal value one at only one arbitrary sampling location insaid block of impulse signal values and signal values zero at othersampling locations; producing a set of frequency coefficients by aorthogonal transformation with said block of impulse signal values;preparing a set of compensating coefficients; converting said set offrequency coefficients to a set of compensated frequency coefficients bymultiplying each compensating coefficient of said set of compensatingcoefficients to a corresponding frequency coefficient of said set offrequency coefficients; producing an output signal value correspondingto said arbitrary location by an inverse transformation of saidorthogonal transformation with said set of compensated frequencycoefficients, said inverse transformation representing signal valuesbetween adjacent signal values of said block of impulse signal values bydecreasing an interval for reproduction of pixel signal values; andapplying said output signal value to said set of coefficients as acoefficient corresponding to said arbitrary sampling location, whereineach compensating coefficient of said set of compensating coefficientscorresponds to a ratio of a frequency response caused by sampling by thepixels of said converted image signal to a frequency response caused bysampling by the pixels of said image signal at corresponding frequency.6. A method according to claim 5 wherein said orthogonal transformationis a discrete cosine transformation.
 7. A method according to claim 5wherein said ratio of a frequency response caused by sampling by thepixels of said converted image signal to a frequency response caused bysampling by the pixels of said image signal at corresponding frequencyis defined as2a/{1+(2a−1)*cos(2*PHI*f*dx)}, where “a” means the aperture ratio ofsaid pixel of said image signal of related direction and is defined tobe from ½ up to 1, “dx” means the size of said pixel of said convertedimage signal of related direction, “f” means said correspondingfrequency and is defined to be less than 1/(4dx), “PHI” means pi, and“*” means multiplication.
 8. A method according to claim 6 wherein saidratio of a frequency response caused by sampling by the pixels of saidconverted image signal to a frequency response caused by sampling by thepixels of said image signal at corresponding frequency is defined as2a/{1+(2a−1)*cos(2*PHI*f*dx)}, where “a” means the aperture ratio ofsaid pixel of said image signal of related direction and is defined tobe from ½ up to 1, “dx” means the size of said pixel of said convertedimage signal of related direction, “f” means said correspondingfrequency and is defined to be less than 1/(4dx), “PHI” means pi, and“k” means multiplication.